Accuracy Of Machine Learning Models Research Articles

Federated learning for predicting compound mechanism of action based on image-data from cell painting

Having access to sufficient data is essential in order to train accurate machine learning models, but much data is not publicly available. In drug discovery this is particularly evident, as much data is withheld at pharmaceutical companies for various reasons. Federated Learning (FL) aims at training a joint model between multiple parties but without disclosing data between the parties. In this work, we leverage Federated Learning to predict compound Mechanism of Action (MoA) using fluorescence image data from cell painting. Our study evaluates the effectiveness and efficiency of FL, comparing to non-collaborative and data-sharing collaborative learning in diverse scenarios. Specifically, we investigate the impact of data heterogeneity across participants on MoA prediction, an essential concern in real-life applications of FL, and demonstrate the benefits for all involved parties. This work highlights the potential of federated learning in multi-institutional collaborative machine learning for drug discovery and assessment of chemicals, offering a promising avenue to overcome data-sharing constraints.

SPLINE OPTIMIZATION OF SOFT CONNECTIVES IN MACHINE LEARNING MODELS

In this study, the problem of limited accuracy of machine learning models using soft logical connectives is investigated. Such connectives have shown their effectiveness in models with fuzzy initial data. On the one hand, the fundamental disadvantage of soft connectives is their non-associativity. On the other hand, the disadvantages of the currently used soft connectives include the loss of monotonicity and the inability to control several factors simultaneously. All these problems have been solved by the authors. We have proposed an approximation of the signum function by a ?1-smooth spline. The first part of the spline is responsible for the curvature of the connective at the diagonal and is adjustable. The second part of the spline is the solution to the optimization problem. We minimized the difference between the connective and the associative connective, the latter in our study was the minimum function. In the resulting solution, the rate of deviation reduction is the highest among known connectives. We have achieved not only a small deviation from associativity, but also the presence of a large domain of exact associativity. This area is up to a third of the volume of all triples of arguments. A comparative analysis of the currently used soft connectives with the constructed model was carried out. It was shown that the spline approximation is able to reduce the influence of all negative factors and is more flexible in setting. Moreover, the constructed spline model allows numerous modifications depending on the factor that requires the most attention for different tasks.

An explainable integrated machine learning model for mapping soil erosion by wind and water in a catchment with three desiccated lakes

Soil erosion by water and wind is a critical challenge for sustainable management of catchments in drylands and accurate spatial information can help in mitigation of its destructive consequences. Here, seven machine learning (ML) models were applied to map simultaneously the water and wind erosions in the Bakhtegan catchment, south Iran, with three dried lakes in its southern part and three dams established in upstream parts of the lakes. The analysis identified 10 and 11 effective variables controlling water and wind erosions, among 20 and 17 potential variables, respectively, via the MARS feature selection algorithm. According to the most accurate ML models (artificial neural network for water erosion, and Cubist for wind erosion), an integrated model was developed to map soil erosion by water and wind simultaneously. Permutation feature importance (PFI) and Shapley additive exPlanation (SHAP) interpretation techniques were employed to explain the model outputs, revealing that 19.7 % of the total area belonged to high and very high susceptibility classes to soil erosion by water and wind. The PFI plot revealed that the slope and wind speed were the most influencing factors for water and wind erosion, respectively. According to SHAP decision plot, slope had the highest contribution on the predictive water erosion model’s output, whereas vegetation cover exhibited the highest contribution on the predictive wind erosion model’s output. Due to climate change and intensified drought during the recent years, as well as due to construction of dams upstream of the lakes, the southern part of the study area was converted to a source of sand and dust storms. The hotspots with severe water erosion are distributed all over the study area, rendering it quite vulnerable to adverse climate change projections.

Random load identification of cylindrical shell structure based on multi-layer neural network and support vector regression

A new-type identification method in the frequency domain by combining a multi-layer neural network and support vector regression is proposed to identify random load of a complex cylindrical shell structure. The kernel function of support vector regression has a great influence on the prediction accuracy of machine learning model, and it is effective to employ the linear function. As the penalty factor is large, the identification accuracy of the Gaussian kernel function is close to the linear kernel function. In the process of random load identification, the prediction accuracy of the neural network using the L-BFGS method is higher than the traditional Adam method. The number of hidden layers of the neural network has little effect on the L-BFGS algorithm, but a great effect on the Adam method. Different levels of noise are introduced to verify the robustness of the machine learning model. Both the support vector regression with linear kernel function and neural network model based on the L-BFGS method have strong robustness. For the noise percentage of 1%, the support vector regression has better prediction accuracy than the neural network, yet the case is contrary for the noise percentage greater than 5%. The verification shows that the neural network model trained by simulation data has better identification accuracy for real load at some frequencies. The load identification method is proposed based on the frequency points which may establish the machine learning model. The mean absolute percentage error shows that the method based on a multi-layer neural network and support vector regression has high identification accuracy.

Essentiality, protein–protein interactions and evolutionary properties are key predictors for identifying cancer-associated genes using machine learning

The distinctive nature of cancer as a disease prompts an exploration of the special characteristics the genes implicated in cancer exhibit. The identification of cancer-associated genes and their characteristics is crucial to further our understanding of this disease and enhanced likelihood of therapeutic drug targets success. However, the rate at which cancer genes are being identified experimentally is slow. Applying predictive analysis techniques, through the building of accurate machine learning models, is potentially a useful approach in enhancing the identification rate of these genes and their characteristics. Here, we investigated gene essentiality scores and found that they tend to be higher for cancer-associated genes compared to other protein-coding human genes. We built a dataset of extended gene properties linked to essentiality and used it to train a machine-learning model; this model reached 89% accuracy and > 0.85 for the Area Under Curve (AUC). The model showed that essentiality, evolutionary-related properties, and properties arising from protein–protein interaction networks are particularly effective in predicting cancer-associated genes. We were able to use the model to identify potential candidate genes that have not been previously linked to cancer. Prioritising genes that score highly by our methods could aid scientists in their cancer genes research.

Bond strength between recycled aggregate concrete and rebar: Interpretable machine learning modeling approach for performance estimation and engineering design

Recycled aggregate concrete has a large potential for application as a sustainable building material. To assist in guiding the engineering design of this low-carbon material, the complex relationship between its bond strength to the rebar and various design parameters was modeled. Comparative studies of multiple machine learning methods have shown that the extreme random tree model demonstrates outstanding performance with root mean square error of 1.982, mean absolute error of 1.202 and coefficient of determination of 0.954. To determine the role of design parameters in performance prediction, importance analysis and sensitivity analyses for parameters were used to explore the complex relationship in the model. The results indicated that rebar type, water-cement ratio, anchorage length, and compressive strength are crucial factors affecting the bond strength between recycled aggregate concrete and rebar. In addition, to further demonstrate the reliability and accuracy of machine learning models, explicit models in some specifications were used for comparative analysis. Due to variability in influencing factors, limited test data, and variability in material properties, the explicit models have limited accuracy for bond strength between recycled aggregate concrete and rebar. Therefore, the interpretable modeling approach can provide an accurate estimate of the bond strength between recycled concrete aggregates and rebar and guide the engineering design of this structure.

An interpretable framework for modeling global solar radiation using tree-based ensemble machine learning and Shapley additive explanations methods

Machine learning techniques provide an effective and cost-efficient solution for estimating solar radiation for solar energy utilization. However, the reported machine learning-based solar radiation models fail to offer comprehensive explanations for their outputs. Therefore, this study aims to tackle this issue by developing machine learning models that are both accurate and interpretable. To achieve the objective, this study evaluated the performance of tree-based ensemble algorithms, using optimized combinations of model input parameters for different climate zones in China. The results showed that the extreme gradient boosting (XGBoost) models demonstrated the highest overall accuracy, model stability, and generalization ability. At the national scale, the developed XGBoost models yielded an average R2, MAE, and RMSE of 0.939, 1.226 MJ/m2, and 1.663 MJ/m2, respectively, showing significant improvements of 2.13–27.78% in RMSE compared to recently reported models. Most importantly, the state-of-the-art SHapley Additive exPlanations (SHAP) technique was integrated with the developed XGBoost models to enhance model interpretability in terms of global and local feature importance, as well as the interaction effects between model features. The results of the SHAP value analysis demonstrated the robustness of sunshine duration in modeling global solar radiation, revealing thresholds where its values undergo a shift from negative to positive effects on model output. SHAP interaction values illustrated the interaction effects among features in the developed solar radiation model, uncovering the model's complex non-linear relationships. Additionally, this study provided explanations for individual instances based on the SHAP method. Overall, this study provided an accurate, reliable, and transparent machine learning model and an enlightening framework for modeling global solar radiation at sites without observations.

An Efficient Deep Learning Approach for DNA-Binding Proteins Classification from Primary Sequences

As the number of identified proteins has expanded, the accurate identification of proteins has become a significant challenge in the field of biology. Various computational methods, such as Support Vector Machine (SVM), K-nearest neighbors (KNN), and convolutional neural network (CNN), have been proposed to recognize deoxyribonucleic acid (DNA)-binding proteins solely based on amino acid sequences. However, these methods do not consider the contextual information within amino acid sequences, limiting their ability to adequately capture sequence features. In this study, we propose a novel approach to identify DNA-binding proteins by integrating a CNN with bidirectional long-short-term memory (LSTM) and gated recurrent unit (GRU) as (CNN-BiLG). The CNN-BiLG model can explore the potential contextual relationships of amino acid sequences and obtain more features than traditional models. Our experimental results demonstrate a validation set prediction accuracy of 94% for the proposed CNN-BiLG, surpassing the accuracy of machine learning models and deep learning models. Furthermore, our model is both effective and efficient, exhibiting commendable classification accuracy based on comparative analysis.

The improvement of ground truth annotation in public datasets for human detection

The quality of annotations in the datasets is crucial for supervised machine learning as it significantly affects the performance of models. While many public datasets are widely used, they often suffer from annotations errors, including missing annotations, incorrect bounding box sizes, and positions. It results in low accuracy of machine learning models. However, most researchers have traditionally focused on improving model performance by enhancing algorithms, while overlooking concerns regarding data quality. This so-called model-centric AI approach has been predominant. In contrast, a data-centric AI approach, advocated by Andrew Ng at the DATA and AI Summit 2022, emphasizes enhancing data quality while keeping the model fixed, which proves to be more efficient in improving performance. Building upon this data-centric approach, we propose a method to enhance the quality of public datasets such as MS-COCO and Open Image Dataset. Our approach involves automatically retrieving missing annotations and correcting the size and position of existing bounding boxes in these datasets. Specifically, our study deals with human object detection, which is one of the prominent applications of artificial intelligence. Experimental results demonstrate improved performance with models such as Faster-RCNN, EfficientDet, and RetinaNet. We can achieve up to 32% compared to original datasets in the term of mAP after applying both proposed methods to dataset which is transformed the grouped of instances to individual instance. In summary, our methods significantly enhance the model’s performance by improving the quality of annotations at a lower cost with less time than manual improvement employed in other studies.

Prediction of corrosion rate for friction stir processed WE43 alloy by combining PSO-based virtual sample generation and machine learning

The corrosion rate is a crucial factor that impacts the longevity of materials in different applications. After undergoing friction stir processing (FSP), the refined grain structure leads to a notable decrease in corrosion rate. However, a better understanding of the correlation between the FSP process parameters and the corrosion rate is still lacking. The current study used machine learning to establish the relationship between the corrosion rate and FSP process parameters (rotational speed, traverse speed, and shoulder diameter) for WE43 alloy. The Taguchi L27 design of experiments was used for the experimental analysis. In addition, synthetic data was generated using particle swarm optimization for virtual sample generation (VSG). The application of VSG has led to an increase in the prediction accuracy of machine learning models. A sensitivity analysis was performed using Shapley Additive Explanations to determine the key factors affecting the corrosion rate. The shoulder diameter had a significant impact in comparison to the traverse speed. A graphical user interface (GUI) has been created to predict the corrosion rate using the identified factors. This study focuses on the WE43 alloy, but its findings can also be used to predict the corrosion rate of other magnesium alloys.

Improving permeability prediction via Machine Learning in a heterogeneous carbonate reservoir: application to Middle Miocene Nullipore, Ras Fanar field, Gulf of Suez, Egypt

Predicting and interpolating the permeability between wells to obtain the 3D distribution is a challenging mission in reservoir simulation. The high degree of heterogeneity and diagenesis in the Nullipore carbonate reservoir provide a significant obstacle to accurate prediction. Moreover, intricate relationships between core and well logging data exist in the reservoir. This study presents a novel approach based on Machine Learning (ML) to overcome such difficulties and build a robust permeability predictive model. The main objective of this study is to develop an ML-based permeability prediction approach to predict permeability logs and populate the predicted logs to obtain the 3D permeability distribution of the reservoir. The methodology involves grouping the reservoir cored intervals into flow units (FUs), each of which has distinct petrophysical characteristics. The probability density function is used to investigate the relationships between the well logs and FUs to select high-weighted input features for reliable model prediction. Five ML algorithms, including Linear Regression (LR), Polynomial Regression (PR), Support Vector Regression (SVR), Decision Trees (DeT), and Random Forests (RF), have been implemented to integrate the core permeability with the influential well logs to predict permeability. The dataset is randomly split into training and testing sets to evaluate the performance of the developed models. The models’ hyperparameters were tuned to improve the model’s prediction performance. To predict permeability logs, two key wells containing the whole reservoir FUs are used to train the most accurate ML model, and other wells to test the performance. Results indicate that the RF model outperforms all other ML models and offers the most accurate results, where the adjusted coefficient of determination (R2adj) between the predicted permeability and core permeability is 0.87 for the training set and 0.82 for the testing set, mean absolute error and mean squared error (MSE) are 0.32 and 0.19, respectively, for both sets. It was observed that the RF model exhibits high prediction performance when it is trained on wells containing the whole reservoir FUs. This approach aids in detecting patterns between the well logs and permeability along the profile of wells and capturing the wide permeability distribution of the reservoir. Ultimately, the predicted permeability logs were populated via the Gaussian Random Function Simulation geostatistical method to build a 3D permeability distribution for the reservoir. The study outcomes will aid users of ML to make informed choices on the appropriate ML algorithms to use in carbonate reservoir characterization for more accurate permeability predictions and better decision-making with limited available data.

Interpretable Machine Learning for Finding Intermediate-mass Black Holes

Definitive evidence that globular clusters (GCs) host intermediate-mass black holes (IMBHs) is elusive. Machine-learning (ML) models trained on GC simulations can in principle predict IMBH host candidates based on observable features. This approach has two limitations: first, an accurate ML model is expected to be a black box due to complexity; second, despite our efforts to simulate GCs realistically, the simulation physics or initial conditions may fail to reflect reality fully. Therefore our training data may be biased, leading to a failure in generalization to observational data. Both the first issue—explainability/interpretability—and the second—out of distribution generalization and fairness—are active areas of research in ML. Here we employ techniques from these fields to address them: we use the anchors method to explain an Extreme Gradient Boosting (XGBoost) classifier; we also independently train a natively interpretable model using Certifiably Optimal RulE ListS (CORELS). The resulting model has a clear physical meaning, but loses some performance with respect to XGBoost. We evaluate potential candidates in real data based not only on classifier predictions but also on their similarity to the training data, measured by the likelihood of a kernel density estimation model. This measures the realism of our simulated data and mitigates the risk that our models may produce biased predictions by working in extrapolation. We apply our classifiers to real GCs, obtaining a predicted classification, a measure of the confidence of the prediction, an out-of-distribution flag, a local rule explaining the prediction of XGBoost, and a global rule from CORELS.

Detection of Autism Spectrum Disorder using Machine Learning: A Review

Abstract: Social interaction and communication impairments are caused by an illness known as Autism Spectrum Disorder (ASD), which has neurological and genetic components. Statistics from the World Health Organization (WHO) show that the number of people with ASD diagnoses is progressively rising. The majority of recent research focuses on data gathering, brain image analysis, and clinical diagnosis; it does not address the diagnosis of ASD using machine learning. Currently, the only techniques available for diagnosing ASD are clinical standardized testing. This results in longer diagnostic times as well as a sharp rise in medical expenses. Machine learning approaches are being used to supplement traditional methods in order to enhance diagnosis precision and time required. The dataset is being subjected to several machine learning approaches, with the aim of constructing predictive models that are contingent on the results. The best accurate machine learning model to identify the disease in its early stages is then found by analyzing each technique according to its assessment metrics.

A hybrid approach of anaerobic digestion model no. 1 and machine learning to model and optimize continuous anaerobic digestion processes

Anaerobic digestion is a promising approach to dispose of biodegradable waste and wastewater, generating biogas as an alternative energy resource. This work proposed a so-called M-CADM1 for continuous anaerobic digestion simulation, which combined the machine learning and anaerobic digestion model No.1 (ADM1). The detailed reaction path and intermediate products in different stages of anaerobic digestion are specified in ADM1. The kinetic parameters were modified by machine learning. The characteristics (elemental composition) of feedstocks were used to predict kinetic parameters. A total of 75 biomass samples were used to establish for machine learning models. Five element contents (C, H, O, N, S), feedstock feed rate, and anaerobic digestion temperature were used as the input. The kinetic parameters were set as output. The sensitivities of 17 kinetic parameters were evaluated. 7 kinetic parameters with the highest sensitivities were selected as ADM1 model inputs by sensitivity analysis. The R2 and RMSE were used as the index to evaluated the accuracy of machine learning model. The best R2 and RMSE reached 0.84 and 0.196. The TIC was used as the index to evaluated the accuracy of M-CADM1. By comparing the simulated value with the experimental value, the accuracy of the overall M-CADM1 expressed by TIC of kitchen waste was 0.036. The organic acid content and pH in the reactor were considered as indicators to study the accuracy and stability of the M-CADM1. Trends in organic acids, free ammonia or hydrogen inhibition, and pH were consistent with experimental continuous anaerobic digestion results.

Predicting acute myocardial infarction from haematological markers utilizing machine learning and explainable artificial intelligence

Myocardial infarction (MI) is the leading cause of human death globally. It occurs when a blockage in an artery prevents blood and oxygen from reaching the heart muscle, causing tissues in the heart muscle to die. This leads to a necessity to develop a method to diagnose MI’s early, preventing further complications such as irregular heart rhythm, heart failure or even cardiac arrest. This research aims to develop a more accurate machine learning (ML) model to help predict acute myocardial infarction (AMI) with a greater degree of accuracy without invasive procedures using additional explainable artificial intelligence (XAI) techniques which will help medical practitioners to better diagnose AMI more precisely. According to the results, the random forest classifier model gave the highest accuracy of 86%. XAI techniques were used to visualize the data and results, and determined white blood cell (WBC) count to be the most crucial feature in classification, followed by neutrophil (NEU) count, neutrophil-lymphocyte (NEU/LY) ratio, platelet width of distribution (PDW) values and basophil (BA) counts. The developed model can help medical practitioners make a more accurate early diagnosis of AMI using readily available hematological parameters, enabling practitioners to provide superior care to a diverse range of individuals.

DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency

The combination of electronic health records (EHR) and medical images is crucial for clinicians in making diagnoses and forecasting prognoses. Strategically fusing these two data modalities has great potential to improve the accuracy of machine learning models in clinical prediction tasks. However, the asynchronous and complementary nature of EHR and medical images presents unique challenges. Missing modalities due to clinical and administrative factors are inevitable in practice, and the significance of each data modality varies depending on the patient and the prediction target, resulting in inconsistent predictions and suboptimal model performance. To address these challenges, we propose DrFuse to achieve effective clinical multi-modal fusion. It tackles the missing modality issue by disentangling the features shared across modalities and those unique within each modality. Furthermore, we address the modal inconsistency issue via a disease-wise attention layer that produces the patient- and disease-wise weighting for each modality to make the final prediction. We validate the proposed method using real-world large-scale datasets, MIMIC-IV and MIMIC-CXR. Experimental results show that the proposed method significantly outperforms the state-of-the-art models.

Analyzing Dataset Annotation Quality Management in the Wild

Abstract Data quality is crucial for training accurate, unbiased, and trustworthy machine learning models as well as for their correct evaluation. Recent works, however, have shown that even popular datasets used to train and evaluate state-of-the-art models contain a non-negligible amount of erroneous annotations, biases, or artifacts. While practices and guidelines regarding dataset creation projects exist, to our knowledge, large-scale analysis has yet to be performed on how quality management is conducted when creating natural language datasets and whether these recommendations are followed. Therefore, we first survey and summarize recommended quality management practices for dataset creation as described in the literature and provide suggestions for applying them. Then, we compile a corpus of 591 scientific publications introducing text datasets and annotate it for quality-related aspects, such as annotator management, agreement, adjudication, or data validation. Using these annotations, we then analyze how quality management is conducted in practice. A majority of the annotated publications apply good or excellent quality management. However, we deem the effort of 30% of the works as only subpar. Our analysis also shows common errors, especially when using inter-annotator agreement and computing annotation error rates.

Qualitative data augmentation for performance prediction in VLSI circuits

Various studies have shown the advantages of using Machine Learning (ML) techniques for analog and digital IC design automation and optimization. Data scarcity is still an issue for electronic designs, while training highly accurate ML models. This work proposes generating and evaluating artificial data using generative adversarial networks (GANs) for circuit data to aid and improve the accuracy of ML models trained with a small training data set. The training data is obtained by various simulations in the Cadence Virtuoso, HSPICE, and Microcap design environment with TSMC 180 nm and 22 nm CMOS technology nodes. The artificial data is generated and tested for an appropriate set of analog circuits and digital cells. The experimental results show that the proposed artificial data generation significantly improves ML models and reduces the percentage error by more than 50% of the original percentage error, which were previously trained with insufficient data. Furthermore, this research aims to contribute to the extensive application of AI/ML in the field of VLSI design and technology by relieving the training data availability-related challenges.

PREDICTION OF CONCRETE MIXTURE DESIGN AND COMPRESSIVE STRENGTH THROUGH DATA ANALYSIS AND MACHINE LEARNING

Concrete is the most used building material in civil engineering. The mechanical properties of concrete depend on the percentage of materials used in the mix design. There are different types of mixture methods, and the purpose of this study is to investigate the mechanical properties of concrete using the mixture method through data analysis. In this case, more than 45 mixture designs are collected to find the estimated mixture design. The estimated mixture design was found by correlation matrix and the correlation between materials of concrete. Moreover, to find the reliability of the compressive strength of concrete through data mining, two models have been established. In this term, Linear Regression (LR), Ridge Regression (RR), Support Vector Machine Regression (SVR), and Polynomial Regression (PR) have been applied to predict compressive strength. In this study, the stress-strain curve of the compressive strength of concrete was also investigated. To find the accuracy of machine learning models, Correlation Coefficient (R2), Mean Absolute Errors (MAE), and Root Mean Squared Errors (RMSE) are established. However, the machine learning prediction model of RR and PR shows the best results of prediction with R2 0.93, MAE 3.7, and RMSE 5.3 for RR. The PR R2 was more than 0.91, moreover, the stress-strain of compressive strengths has been predicted with high accuracy through Logistic Algorithm Function. The experimental results were acceptable. In the compressive strength experimental results R2 was 0.91 MAE was 1.07, and RMSE was 2.71 from prediction mixture designs. Finally, the prediction and experimental results have indicated that the current study was reliable.

Empowering prediction of miRNA–mRNA interactions in species with limited training data through transfer learning

MicroRNAs (miRNAs) play a crucial role in mRNA regulation. Identifying functionally important mRNA targets of a specific miRNA is essential for uncovering its biological function and assisting miRNA–based drug development. Datasets of high-throughput direct bona fide miRNA–target interactions (MTIs) exist only for a few model organisms, prompting the need for computational prediction. However, the scarcity of data poses a challenge in training accurate machine learning models for MTI prediction. In this study, we explored the potential of transfer learning technique (with ANN and XGB) to address the limited data challenge by leveraging the similarities in interaction rules between species. Furthermore, we introduced a novel approach called TransferSHAP for estimating the feature importance of transfer learning in tabular dataset tasks. We demonstrated that transfer learning improves MTI prediction accuracy for species with limited datasets and identified the specific interaction features the models employed to transfer information across different species.